# Knowledge distillation

This is the quantized version of OpenBuddy-R1-0528-Distill-Qwen3-32B-Preview0-QAT. With quantization technology, the model can run more efficiently under different hardware conditions.

An 8B-parameter language model merged using the SLERP method, combining the strengths of DeepSeek-R1 and Nemotron-8B models

Distill-Any-Depth is a new SOTA monocular depth estimation model trained using knowledge distillation algorithms.

A state-of-the-art bilingual language model based on the Llama architecture, specially optimized for Korean understanding and generation while maintaining strong English capabilities.

KOALA-Lightning-1B is a knowledge distillation model based on SDXL-Lightning. It achieves efficient text-to-image generation by compressing the U-Net structure, with a parameter scale of 1.16B.

A distilled version of ProtGPT2, compressed into a more efficient small model through knowledge distillation, maintaining performance while improving inference speed

Implementation of the SPLADE++ model optimized for industrial scenarios, balancing retrieval quality and efficiency, and supporting document expansion and sparse representation learning

MMLW (I Must Get Better Messages) is a neural text encoder for Polish, optimized for information retrieval tasks.

MMLW (I Must Get Better News) is a Polish neural text encoder optimized for information retrieval tasks, capable of converting queries and passages into 768-dimensional vectors.

BK-SDM is a stable diffusion model compressed through architecture compression, used for efficient and general text-to-image synthesis. It achieves lightweight design by removing residual and attention blocks in the U-Net.

LEALLA is a collection of lightweight, language-agnostic sentence embedding models supporting 109 languages, distilled from LaBSE. Suitable for multilingual sentence embeddings and bilingual text retrieval.

LEALLA-small is a lightweight, language-agnostic sentence embedding model that supports 109 languages, suitable for multilingual sentence embedding and bilingual text retrieval tasks.

A lightweight BERT model for the Italian legal domain built using knowledge distillation technology, featuring only 4 Transformer layers

The efficient SPLADE model is used for paragraph retrieval. It adopts a dual-model architecture to handle query and document inference respectively, and performs excellently on the MS MARCO dataset.

KoMiniLM is a lightweight Korean language model designed to address latency and capacity limitations of large language models in practical applications.

This is a TinyBERT model optimized for German, created by distilling the BERT base cased model, suitable for natural language processing tasks.

A lightweight text evaluation model based on knowledge distillation, using a small student model to mimic the scoring behavior of a large teacher model

This model is a BERT-Base fine-tuned for QA tasks, using 80% 1x4 block-sparse pre-training combined with knowledge distillation.